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Abstract 

Using an effective Lagrangian approach, we perform a model-independent analysis of the inter- 
actions among electroweak gauge bosons and the third generation quarks, i.e. the Wtb, Ztt and 
Zbb couplings. After one imposes the known experimental constraint on the Zb^L coupling, we 
show that the electroweak SU(2)l x U(1)y symmetry of the standard model specifies the pattern 
of deviations of the Zt^ti and Wt^bi couplings, independent of underlying new physics scenar- 
ios. We study implications of the predicted pattern with data on the single top quark and Ztt 
associated production processes at the Large Hadron Collider. Such an analysis could in principle 
allow for a determination of the Wtb coupling without prior knowledge of |Vt&|, which is otherwise 
difficult to achieve. 
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I. INTRODUCTION 



It is widely accepted that interactions of the third generation quarks, the top and bottom 
quarks, offer a window into possible new physics beyond the standard model (SM) of particle 
physics. Since the top quark mass is close to the Fermi scale v = (v^Gp) -1 / 2 m 246 GeV, its 
interactions are thought to be sensitive to the mechanism of electroweak symmetry breaking. 
Indeed, in the majority of models beyond the SM, deviations are predicted from the SM in the 
top quark interactions, especially in their couplings with the electroweak gauge bosons, such 
as the Wtb, Zbb, and Zti couplings. Interactions of the top quark have yet to be measured 
precisely, allowing possible room for deviations from the SM. Experimental constraints on the 
bottom and top quark couplings with the W and Z bosons provide an interesting contrast: 
the bottom quark left-handed coupling Zb^bi is determined precisely by the measurements 
at LEP I and II whereas the Zt^tp coupling is virtually not measured so far. The 
Wtibp coupling was confirmed only recently in single top quark production at the Fermilab 
Tevatron 2j, y|, |4j. It will be probed further at the CERN Large Hadron Collider (LHC). 

In this work we investigate the range of possible deviations of the couplings of the top and 
bottom quarks with the electroweak gauge bosons by exploiting the contrasting experimental 
constraints mentioned above. Adopting an effective Lagrangian approach [51], we parametrize 
the effects of new physics in terms of higher dimensional operators constructed from SM 
fields: CiOi/A n ~ 4 , where n is the mass dimension of the operator 0j, A is the scale of new 
physics, and q is a numerical coefficient assumed to be of order unity unless otherwise 
specified. Such an approach is valid when all new particles are heavier than the Fermi scale 
and whose effects can be integrated out of the effective theory, as we assume. However, this 
assumption does not preclude the possibility that new particles at the TeV scale could be 
produced directly and observed at the LHC. 

Within the general context in which we work, we demonstrate that the SU(2)l x U(1)y 
symmetry of the SM yields correlations among the possible deviations of the couplings 
Zbibi, ZtitL, and Wtj))h- Once the stringent experimental bound on Zbjjoi, is taken into 
account, a unique prediction follows on the size of the ZtrfL and Wtibi couplings. The 
prediction is a striking manifestation of SU{2)l x U(1)y invariance, and it is independent 
of the underlying new physics at the electroweak scale. 

After an exposition of the general operator analysis in Section II, we use existing exper- 
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imental constraints to show that deviations of the Wtb and Ztt couplings from their SM 
values can depend on only two parameters, Tl and Tr, and we present the allowed ranges of 
these parameters for different new physics cutoff scales and Higgs boson masses. In Section 
III, we survey the landscape of new physics models which may be expected to modify these 
couplings. The prospects that Tl and Tr can be determined better from data from the 
LHC are addressed in Section IV where we focus on single top quark and Ztt associated 
production. Our results from Section II show deviations from the SM of the single top quark 
total and differential cross sections depend only on Tl, whereas the cross section for Ztt 
associated production is influenced by both Tl and Tr. 

The presence of contributions from two operators in Ztt associated production means that 
observation, or at least a bound on the effects of new physics in this process is in principle 
possible provided one looks sufficiently differentially into the distributions of particles in the 
final state. We began our study expecting to show that spin correlations in the final state 
would offer significant advantages, particularly those between the top quark spin and the 
decay lepton from t — ► bW + (—> However, as we conclude from a detailed simulation 

in Sec. IV, experimental cuts distort the most telling distributions and appear to leave a 
sample of effectively unpolarized top quarks. This result, while disappointing, does not 
appear to have been established previously. Turning adversity to advantage, we remark that 
the difficulty in finding evidence for new physics in the Ztt final state may be interpreted 
as good news for new physics searches. The associated production of Ztt is a dominant 

n q 

background for new physics models [6|, 7J. If a new physics model predicts the production 
of highly polarized top quark pairs, then one might be able to see evidence above the 
effectively unpolarized dominant background. Among the final state distributions in Ztt 
associated production, we confirm that the opening angle between the two charged leptons 
from the Z boson decay, A(f>(£ /+ £'~), appears useful for limiting the couplings Tl and Tr, 
as is pointed out in Ref. 8J. 

We also comment that in single top quark production Tl is always multiplied by the 
CKM matrix element \Vtb\, which is very close to one in the SM. Therefore if both Tl and 
Tr could be extracted from Ztt production through the total and differential cross-sections, 
one could infer the value of \V t b\ from measurements in singlet top quark production. 

We present the results of our analysis as a set of two-dimensional plots of the deviation 
from the SM of the associated production cross section baztt versus the deviation of the 
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single top quark cross section Sa t for integrated luminosities of 300 fbT 1 and 3000 ffcT 1 at the 
LHC. These expectations are contrasted with estimates for a high energy electron-positron 
linear collider (LC). A comparison is also shown with predictions based on recent models of 
new physics including a top-prime model a right-handed t' model [loj], and a model with 
sequential fourth generation quarks that mix with the third generation Our analysis 
suggests that about 3000 ftT 1 at the LHC would be needed to improve model independent 
constraints on Tl and Tr to the same level achievable at a 500 GeV LC with lOOfb -1 
of integrated luminosity. On the other hand, 300 ftT 1 at the LHC would be sufficient to 
delineate which new physics models could be possible. 

In addition to the study in Ref. |s| of Zti associated production mentioned above, other 
earlier work related to ours includes the study in Ref. [12|] of the prospects for measuring the 
Wtb coupling at a linear collider as a test of different models of new physics. The authors 
investigate tt production and single top quark production, as we do, but they do not focus 
on the correlations. 



II. OPERATOR ANALYSIS AND EXISTING CONSTRAINTS 

We begin with a general assumption that effects beyond the SM are described by a set 
of higher dimensional operators made out of the SM fields only. Once the (approximate) 
symmetry of the SM is assumed, these operators start at dimension six [5] . A complete list of 
operators is presented in [ljj, whose notation we follow. Therefore the effective Lagrangian 
we work with is of the form 

C eff = C SM + ^ £ (CA + h - C -) + (Xs) ' W 

where the coefficients q's are numerical constants parameterizing the strength of the non- 
standard interactions. The excellent agreement between the SM expectations and data 
indicates that deviations from the SM are small. Hence, when computing the effects of new 
operators we can restrict ourselves to the interference terms between C$m and the operators 
Oi, i.e. we can work to first order in the coefficients q. 
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A. Operator Analysis 



Three types of dimension-six operators contribute to the Wtb, Zbb, and Zti couplings: 
1) operators involving scalars and vectors, 2) operators involving fermions and vectors, and 
3) operators involving vectors, fermions, and scalars. We discuss all three types in turn in 
the following. 

Operators involving scalars and vectors enter through the self-energy of the electroweak 
gauge boson: 0+ w = §(0ty) W^W 1 ^, 0^ B = |0V) B^B^, and O wb = (0V0) W^B^, 
where <fi denotes the SM scalar Higgs doublet, W* and B^ v are the field-strength tensors 
for the SU{2)l and U(1)y gauge bosons respectively, and r 1 = a 1 /2 is the usual SU(2)l 
generator in the fundamental representation. Operators 0<pw and O^b arise only after new 
physics is integrated out at the loop level [l^]. Their corrections to the self-energy are of 
order (1/167T 2 ) x (v 2 /A 2 ), where v is the vacuum expectation value of the Higgs. Operator 
Owb can be generated by tree- level exchange of new particles, but its contribution is related 
to the 5* parameter. It is highly constrained by precision electroweak data, which requires 
cwb ~ O(10~ 2 ) [jj]] for A ~ 1 TeV. Thus, all three operators are effectively suppressed by 
a loop factor for new physics at about the 1 TeV scale, and we neglect them in this study. 

Operators of the second type necessarily have two fermions carrying one gauge-covariant 



derivative and one field strength [l3|, such as O q w = i{<l' Tl l^D v q) W*. These operators give 
a correction of order p 2 /A 2 to the couplings of interest here, where p is the typical momentum 
scale in the process and can be taken to be the Fermi scale p ~ v . However, such operators 
correspond to vertices with only three legs. They appear only once new physics is integrated 



out at the loop level Il7j ] . Their natural size is again of order 



(1/16tt 2 ) x (v 2 /A 2 ). A 



similar conclusion can be obtained from naive dimensional analysis 18|, since each derivative 



in the operator carries an extra 47r suppression. Therefore, we do not consider operators of 
the second type here. 

Operators of the third type can be generated both at the tree-level and at the loop 
level. The loop- induced operators, such as qa^T^RCpW^, qa^rHR^W^, qa^tRCpB^, and 
qa^bRipB^, are not included in our analysis as they are suppressed by small coefficients of 
order 1/167T 2 [yj]. We focus our attention on the tree- level induced operators of the third 
type throughout this paper. 
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The dimension- six operators of the third type are 



Of q = i (tfDj) ifffq) , (2) 

Of q = i (fr'Drf) (g 7 Vg) , (3) 

0# = i (tfD^) (t Rl H R ) , (4) 

= i (tfD^) (b R rb R ) , (5) 

o H = UUD^) (i R Yb R ) , (6) 



where is the gauge-covariant derivative; q is the left-handed top-bottom SU(2)l doublet 
q = &£,); t R (b R ) are the corresponding right-handed isosinglets; and e = icr 2 is the 
two-dimensional antisymmetric tensor. Corrections from these operators are of order t> 2 /A 2 
and could be generated from integrating out new physics at the tree-level. For example, 
the operator 0^ q can be induced by a heavy t quark mixing with the top quark, which is 
present in many theories beyond the SM. 

It is worth mentioning that equations of motion can be used to turn the Owb operator 
into type 3) operators that are universal in flavors Thus, the statement we are about 
to make applies to contributions from Owb as well. 

Upon symmetry breaking (0) = v/y/2. The set of operators of the third type generate 
the following corrections to the couplings Wtb, Zti and Zbb: 

(3) 2 2 

Owto = Cj ^^K^b L - ^to=W+i#fb R + h.c, (7) 

U Ztt - T7j ^ Z ^Ll l L 72 ^ Z /Ar7 *R> (°J 

CzfeS = — - — ^ Z^b L ^b L - J jr~ — ^ Z„b R j"b R , (9) 

where #2 and <?i are the coupling strengths of the SU(2) and U(l) gauge interaction, respec- 
tively. 



B. Existing Constraints 



Among the five operators listed above, O^ is tightly constrained by recent data on 
the rare decay of b -> 57, -0.0007 < < 0.0025 



21 



22 



23j, provided there is 



no accidental cancellation with contributions from other new physics effects, such as those 
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produced by the four-fermion operator bstt. 1 As a result, effects from are small and 
are not considered further. 

For the purpose of our analysis, the most useful experimental constraints on the five 
operators of the third type are the precise measurements of R b and Ap B at LEP II 
These bound the Zbb coupling. The measured value of the ZbjJ>L coupling agrees with the 
SM prediction at the 0.25% level. It enforces the relation 

cf q + eg a 0, (10) 

which, in turn, implies that the deviations in the Wt^bi and Zt^ti couplings are controlled 
by the same parameter eg ~ — eg. In other words, the SU{2)l x U(1)y symmetry of the 
SM predicts a certain pattern in the deviations of the electroweak gauge boson couplings to 
the third generation quarks that is independent of the possible new physics beyond the SM. 



Certain subgroups of the custodial symmetry [24| which protect the p(= mw/mz cos 9w 



parameter can also preserve the Zbjjat coupling in the SM 25], resulting in eg + eg = 



exactly. This result is obtained if the top and bottom quarks are embedded in suitable 
representations of SU{2) L x SU{2) R whose diagonal group SU{2) V serves as the custodial 
symmetry. If one implements the custodial symmetry in this way, the Zt R t R coupling is 



also protected, = 25]. In this work we are not concerned with the underlying reason 
for the smallness in the deviation in the Zbjj)^ coupling. We take Eq. (ITOl) as an empirical 
statement, allowing us to be model-independent. A recent study of top compositeness and 
the third generation couplings to electroweak gauge bosons within the the framework of 



Ref. 



25] can be found in Ref. 26]. 



After Eq. ( flOl) is imposed, the Wtb and Ztt couplings depend on only two unknown 
parameters: 

O wtb = T L W + i L Yb L + h.c. , (11) 

o Z u = ^-z^ {2T L t Ll n L + t r t Rl n R ) , (12) 

where Tl = cgf 2 /A 2 , F R = — c^w 2 /A 2 , and c w = cos6 w is the cosine of the Weinberg 
angle. Notice the relation between the coefficients of the left-handed neutral and charged 



1 This operator can be generated, for example, by exchanging a heavy W' vector boson. 
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currents 2 : 

9ztt = ^9wtb = ^ r L. (13) 

This equation states the pattern of deviations predicted by the electroweak symmetry of the 

SM, after the stringent constraint on Zbpbi is imposed. Notice further that the right-handed 

coupling Zbftbji does not enter the stated correlation in the Wtb and Zti couplings, leaving 

room for the interesting possibility that bn could be (partially) composite. As it turns out, 

a positive shift of the Zb^n coupling, say Sg ZbR i R ~ +0.02, would explain the 3<r deviation 

in the forward-backward asymmetry Ap B measured by the LEP and SLAC Large Detector 
Q 

experiments |28l] . 

Within the low-energy effective theory, JF L and Tr in Oztt induce one-loop corrections 



to the p parameter and the Zbb vertex, which are associated with the observab 
summarized in Ref. j^]. The pure SM one-loop contributions to e\ and e& are 



es ei and e& 



sm 3GVm t 3Gpm w 2 / m H 



, . - _ tan 2 ^log — , (14) 

4 M = (15) 

where m H (m w ,m z ,m t ) denotes the mass of Higgs boson (W-bosoia, Z-boson, top quark), 



respectively. The contributions from the anomalous couplings JF L and Tr are 



2l| 



*■ = w^*-^ 5 • (i6) 



mfGp ( r 1 \ /A 2 
V 2J^ L - -^r In —= 
2y/2n 2 V 4 H J \m 2 t 



6e b = ^-(2F L -~F R )ln{- 2 ), (17) 



where A is the cutoff of the low-energy effective theory and is taken to be the scale of new 
physics. Notice that ef M depends on mn, which implies that the allowed region of Tl and 
Tr will depend on as well as on A. 



Using the experimental results 



29|, 



4.4 x 10~ 3 < eT P < 6.4 x KT 3 , 
-6.2 x 10~ 3 < e e b xp < -3.1 x 10" 3 , 

we plot the experimental constraints on JF L and Tr i n Fig- E assuming the absence of 
contributions from other operators. It is important to contrast the allowed region for JF L 



2 A similar relation is pointed out in Ref. [27| in the context of an electroweak chiral Lagrangian, even 
though the implication of such a relation was not studied. 
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(a) A = TeV 



£ 0.00 




FIG. 1: Allowed regions of J~l and Tr (a-c) and of ; and c^t (d-f) for different NP cutoff scales 
A and SM Higgs boson masses mjj. 



and Tr with the expected sizes CiV 2 /A 2 based on the power counting discussed earlier in 
Section III Al As expected, the constraints from one-loop corrections do not require the 
magnitudes of and to be smaller than their 0(1) natural sizes, see Fig. [IJd-f). 

Last, we comment on other new physics effects which could modify the above bounds. It 
is worth mentioning that many tree-level induced operators can also contribute to Z — > bb 
decay, diluting the constraints we derived above without breaking the correlations between 
the Wtb, Zbb and Zti effective couplings. For example, four-fermion operators qqbb would 
affect Z — > bb decay via seagull type loop diagrams. An exploration of the constraints (or 
correlations) among all tree-level induced operators from the p parameter and LEP Z — > bb 
data is highly desirable, but it is beyond the scope of this paper and will be presented 
elsewhere. Therefore, we take the above constraints as an indication of the magnitude of the 
effective couplings but do not limit ourselves to the above parameter space in the subsequent 
collider analysis. 
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III. NEW PHYSICS SCENARIOS 



Having established in a model independent fashion the possible pattern of deviations in 
the Wtb, Zbb, and Zti couplings, we offer a brief survey in this section of new physics models 
that give rise to such deviations. The treatment here is by no means complete and serves 
only to exemplify the generality of the operator analysis. 

As discussed in the previous section, we are interested mainly in deviations arising from 
non-oblique corrections, which are much less constrained experimentally. The simplest possi- 
bilities for such corrections are new particles mixing with the SM top quark, the SM bottom 
quark, or both. 

The possibility of introducing additional bottom-like quarks (b'), which mix with the 
SM b quark, to resolve the discrepancy of the forward-backward asymmetry of the b quark 
(A b FB ) is investigated in Ref. [3l|. However, because of the stringent constraint on Zb^bi, the 
mixing could be significant only in the right-handed sector, implying negligible deviations 
in the WtibL coupling. Since the Wtnbn coupling is already severely constrained by b —>■ 57 
data, as mentioned previously, we do not expect this class of models to produce significant 
deviations in the Zti and Wtb couplings. 

A custodial 0(3) symmetry would protect simultaneously the p parameter and the Zbibi 



coupling 



25]. In this case significant mixing of b' with the SM b quark in the left-handed 



sector is possible, provided additional top-like quarks with appropriate quantum numbers 
are also present so as to cancel the V contribution to Zbibi- Explicit examples are found in 



Refs. 



32 



33j,|3j. 



The third class of models we discuss has an exotic top-like (f) quark which could mix 
with the SM top quark. Such a scenario appears quite often in theories beyond the SM, 
especially those attempting to address the quantum stability of the Higgs boson mass. As 
is well-known, within the SM the largest contribution to the one-loop quadratic divergence 
of the Higgs boson mass comes from the top quark. Therefore, if the Higgs boson mass 
is to be at the order of a few hundred GeV without significant fine-tuning, a new particle 
(i.e., the t! quark) must be present to cancel the top quark contribution in the quadratic 
divergence. This is the case in models where the Higgs boson arises as a pseudo-Nambu- 



Goldstone boson such as in Little Higgs theories 



351 ]. the holographic Higgs model 



3J|, the 



twin Higgs model 37]] , and so forth. On the other hand, there are also models with a t' quark 
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which may not deal with the electroweak hierarchy problem explicitly Notable examples are 



those with flat 



38] or warped 39J extra dimensions where the SM field propagate. In this 



case the t' quark is nothing but the Kaluza-Klein partner of the SM top quark, taken to be 
the zero mode. Very recently a t' quark was also invoked to explain a possible experimental 
excess at the Tevatron 

Having surveyed some candidates for new physics beyond the SM, to which our model in- 
dependent analysis applies, we address in the next section how to constrain the two unknown 
parameters Tl and JFr at the LHC. 



IV. MEASUREMENTS OF THE Wtb AND Ztt COUPLINGS AT THE LHC 

In this section we discuss how the Wtb and Ztt couplings could be measured at the LHC. 
The former can be measured most directly in single top quark production and the latter 
in the Ztt associated production. We first consider extracting Tl and Tr from total cross 
section measurements. Then we look beyond the total cross section and study differential 
distributions in the decay products of Ztt associated production. 



A. Current Experimental Bounds 

Single top quark events result from the i-channel process (ub — > dt), the s-channel process 
(ud — > tb) and Wt associated production (bg — > tW~). The distinct kinematics of each of 
these processes allows differentiation among their contributions. Observation of single top 

n fi 

quark production was reported recently by the CDF and DO collaborations |3j, |4(. These 
results provide the first direct measurement of the product of the Wtb coupling (gwtb) and 
the CKM matrix element V tb . CDF quotes \V tb \ = 0.91 ± 0.11(stat + syst) ± 0.07(theory) 
and a limit |V t6 | > 0.71 at the 95% C.L. [for m t = 175 GeV]. DO obtains \V tb {l+F L )\ = 
1.07 ±0.12(stat + syst + theory) and a limit of \V tb \ > 0.78 at the 95% C.L. [m t = 170 GeV]. 
The limit on V t b is derived under assumption of gwtb = Qwtbi = 0. However, the 

bound can also be translated into a bound on Tl if one assumes the unitarity of the 3x3 
CKM matrix element (|V#,| = 1). Taking the DO result at face value and inserting |Vt&| = 1, 
we see that these data could allow Tl — 0(0.1), roughly twice the size shown in Fig. [TJ 
On the other hand, obvious limitations have precluded measurements of the Ztt coupling 
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(gztt) thus far. There was insufficient center-of-mass energy at LEP to produce a top quark 
pair via e + e~ — > j/Z — > it. At hadron colliders, it production is so dominated by the QCD 
processes gg — *> ti and qq — > g — > ti that the signal from g zt j via qq — > j/Z — > ££ cannot 
be extracted. However, one might be able to measure the gztt coupling via the process of 
gg -> Ztt. 



The sensitivity to non-standard Wtb coup 



is investigated in several papers 
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23, 



40, 



41 



42 



;he LHC via single top quark production 



43 



44 



451 ]. while a study on extracting 



46|. Such 



the Ztt coupling in Ztt associated production at the LHC appears in Refs. 
measurements would also be a focus of study in a future high energy electron-positron linear 
collider (LC). 

It is worth pointing out that, since measurements of single top quark production can only 
probe the combination |K&(1 + it would be desirable to measure Tl independently of 
\V t b\, which could be achieved by utilizing Zti associated production. 



B. Cross sections for single top quark and Ztt associated production 

Because the operator Owtb is proportional to Tl, only the overall normalization of the 
single top quark cross section is affected, and, except for normalization, the final state 
differential distributions are insensitive. A measurement of T L requires a very precise mea- 
surement of the total cross section. The coupling Tt also affects top quark decay, but it 
does not change the top quark decay branching ratio, i.e. Brit — > W + b) = l. 3 On the other 
hand, the Zti amplitude involves both T L and Tr, and it is conceivable that differential 
distributions in the decay products of Zti production would have different sensitivity to Tl 
and respectively. We will look at two possibilities: the opening angle A<ft(£ + ,£~) from 
the decay of Z — > £ + £~ and the spin correlation between the top quark and the Z decay 
products. 

The inclusive cross sections for single top quark and Zti associated production at the 
LHC are: 



1 + 2T L + 25V tb + O (t\, 6V t l)} , (18) 



3 Whatever Tl contributes to the matrix element of the decay is canceled by its modification to the top 
quark decay width. 
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FIG. 2: (a) Contours of the deviation of the Ztt production cross section in the plane of J~l and 
F R , where 5a Z tt = -20% (red dotted), -10% (red dashed), 0% (black), and 10% (blue dashed); 
(b) Correlation of 5a ztt an d Sat for different Tr. The red (green) band is the 3 standard deviation 
statistical variance of the Ztt production cross section at the LHC with an integrated luminosity 
of 10 fb" 1 (100 fbT 1 ), respectively. 



oztt 



a 



ztt 



1 + 4AF L - l.5F R + O (t\, T% T l T r 



(19) 



where of and a zt ^r denote the SM cross sections for single top quark production and Ztt 
production, respectively. We include the possibility of a non-unitary CKM matrix element 
SV tb = \V tb \ {exp) - \ Vt b \ iSM) . From Eq. 

we see immediately the possibility of extracting 

5V t b from 

5V tb = -0.23Sa zt - t + 0.5Sa t - 0.34^, (20) 

which is not possible from measurements of single top quark production alone. In the 
above, Tr could in principle be measured from differential distributions in Ztt associated 
production, as is discussed in detail below. 

Since new physics contributions to the Wtb, Ztt, and Zbb couplings are of the or- 
der v 2 /A 2 ~ l/(167r 2 ) for A ~ 1 TeV, we can safely ignore interference effects between 
new physics and SM one-loop contributions in the total cross section. Therefore, the SM 



quantities Jii Eq . 
Refs. 



47 



48 



49, 




51 



are understood to 
5 



52 



53 



55 



56 



:>e evaluated at one-loop level, as calculated in 



57 



58 



59 



60]. 



We define the deviation of the cross sections from the SM predictions as 5a = 
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(er — er°) /a . The contours of Saztt m the plane of Tl and JFr are shown in Fig. [2(a). 
The ranges of Tl and Tr in this figure are consistent with the allowed regions shown in 
Fig. [TJ All the contours are straight lines since we keep only the interference term. The 
anticipated deviation Saztt is as large as 20%. We observe that in Eq. ffTU]) Tl and Tr con- 
tribute with opposite signs, implying partial cancellations in the new physics contributions. 
In fact, if Tl/Tr —1/3 then Saztt — 0, as one may see in the black-solid curve of Fig. 12(a). 
On the other hand, if Saztt > we could infer immediately that Tl/Tr > 1/3, and vice 
versa. The correlation between the deviations of the cross sections for various values of Tr 
is revealed in Fig. [2(b), where SV t b = is assumed. When the right-handed Ztt coupling is 
not modified by any new physics, say Tr = 0, then 

Saztt = 2.2Sa u (21) 

as illustrated by the black-solid curve in Fig. [2(b). Non-zero values of SVtb increase the 
intercept on the y-axis of all the lines in Fig. [21 as SVtb is always negative. The solid bands 
in the figure are the 3 standard deviation statistical variations of the SM total cross section 
for different luminosities. Given large enough deviations in both Ztt and single top quark 
production, one could determine the values of Tl and Tr uniquely if = 1. (For example, 
one can determine Tr by substituting Tl derived from the single top quark measurement 
into the Ztt measurement.) If one relaxes the constraint on V t b then the two measurements 
merely yield a relation between Vtb and Tr, see Eq. ( |20|) . Nevertheless, it may be possible 
to extract Tr independently from differential distributions in Ztt, as these are sensitive to 
the anomalous couplings. If one takes SV t b = as an assumption, then Tl and Tr could be 
over-constrained by measurements of the total and differential cross sections, allowing for a 
check on the relation in Eq. (fl3l) . In the rest of this study we simply assume SV t b = unless 
otherwise specified. 

Next we consider two differential distributions in Ztt associated production: the opening 
angle A(p(£ + £~) in Z — > £ + £~ and spin correlations defined below. In studying the impact 
of the two effective couplings Tl and Tr on the top quark spin correlations, we look at the 
final state 

qq/gg - ttZ, Z - £'+£'', t - 6W+(- £ + u), i - bW~(-^ jj), (22) 

where £(£') denotes a charged lepton, b a bottom quark jet, and j a light quark jet. The 
collider signature is £' + £'~ £ + bbjj plus missing energy Ifi?- The backgrounds to the tri-lepton 
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LtbZ)jj 
8j, both 



final state considered in Ref. [8| are from (tbZ + ibZ) + X production, e.g. (tbZ 
and (tbZ + ibZ)£i>, and from non-resonant WZbbjj production. As evaluated in 
background rates are one order of magnitude less than Zti production, and we do not 
consider them in this study. Heavy flavor contributions to tri-lepton final states are examined 
in Ref. [61 1. 



We begin with the initial expectation that the spin of the top quark in Zti production 
is a good discriminating variable. The potential for measuring the top quark polarization 
in Zti production is not studied yet in the literature, to the best of our knowledge. The 
charged lepton in top quark decay can be used to measure the top quark spin and thereby 
determine the anomalous couplings. To investigate this possibility quantitatively, we perform 
a quantitative numerical simulation of Zti associated production including the decays of the 
Z boson and top quarks. In addition to the usual helicity basis, we propose and examine a 
new 'optimal' basis that improves the measurement of top quark polarization. 

Another differential observable is the opening angle in the transverse plane between the 
two charged leptons from Z boson decay, A<p(£' + £'~), also investigated in the study of Refs. si, 



461 ]. In this work we show explicitly that this azimuthal opening angle is a better observable 



for measuring the anomalous Zti couplings than the top quark spin because it is relatively 
insensitive to kinematic cuts. The correlation between Zti and single top quark production 
is exploited at the end of this section. 

We mention in passing that Zti associated production is an important SM background 
in searches for possible new physics, especially when the Z boson decays into neutrinos, 
resulting in missing transverse energy in collider detectors. Examples are pp — > tt — > tixx m 
the minimal supersymmetric standard model (MSSM) where t and x denote the top squark 
and the neutralino (Wino like), and pp — » T_T_ — > tiAuAn in the Little Higgs theories 



with T-parity (LHT) [62, |63|, |64|, |65|, |66|, |67|, where T_ and A H are the T-odd top quark 



and photon partners, respectively. In the LHT, the Ah — t — T- coupling is predominately 
right-handed polarized. In the MSSM the x ~ t ~ t coupling depends on the t mixing. The 
top quark polarization then will be a key to distinguish or to provide information on various 
models. 



15 



C. Monte Carlo Simulation 



We perform a Monte Carlo simulation of Ztt production at the parton level, sufficient for 
our purposes. We do not include SM one-loop contributions in our simulation of the differ- 
ential decay rate. These effects should certainly be taken into account when one attempts 
to analyze real data. 



1. Event Reconstruction 



To mimic detector capability, we require the transverse momentum of the charged lepton 
and jets (including both h and j) to satisfy the following basic cuts: 

Vt > 15 GeV, \r] £ \ < 2.5, p b T > 20 GeV, \r] b \ < 2.5, 
p> T > 15 GeV, \r]j\ < 2.5, E T > 20 GeV, 

AR(j, j) > 0.4, AR(j, I) > 0.4, AR(j, b) > 0.4, AR(b, b) > 0.4. (23) 



Here AR = y (A0) 2 + (Ar/) 2 is the separation in pseudo-rapidity-azimuth space, and ~$t is 
the missing transverse momentum originating from the neutrino which escapes the detector. 



In this study we adopt the p^-dependent 6-tagging efficiency defined as 

e b = 0.57 x tanh ' ' 1 



68) 



35 GeV J 

We smear all final state parton momenta by a Gaussian distribution with 

AE 50% 



E ^E 1 

where E is the energy of the observed parton, and the resolution in energy is assumed to 
be 50%v^E'- We also require that there be a same flavor, opposite-sign charged lepton pair 
with invariant mass near the Z resonance, 

\mi'+i'- — m z\ < 10 GeV. 

As a result of this final state signature requirement, Ztt production as observed is insensitive 
to ti'-f production, where 7 denotes a virtual photon. 

To study spin correlations, one must reconstruct the H^-boson pair and the top quark 
pair. The hadronically decaying W could be reconstructed from the invariant mass of the 
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FIG. 3: Invariant mass distribution of the true top quark (black) and the reconstructed top (red). 
An integrated luminosity of lOOfb -1 is used here. 

two light jets, while the leptonically decaying JV-boson is reconstructed from the final state 
electron and the observed missing transverse energy fix- The lack of information about 
the longitudinal component of the neutrino momentum {p u z ) is addressed by requiring the 
invariant mass of the electron-neutrino system to be equal to the mass of the IV-boson. 
This additional constraint yields two possible solutions for p z , and typically, both of them 
are physical solutions for a signal event. We follow the prescription in Ref. 69| to choose 



the solution which has the smaller \p v z \. This method picks the correct p v z in about 70% 
of the events passing the above basic cuts. We find no physical solution for quite a few 
events due to the detector smearing effects. To recover these events, we generate a Breit- 
Wagner distribution around mw and use the generated mass to derive p z . About 7% of the 
remaining events do not exhibit a physical solution and are not included in our analysis. 

To reconstruct the top quark, we combine the reconstructed W^-boson with the 6-jet from 
the top quark decay. The challenge in this case is to identify the correct jet. To this end 
we make use of the top quark mass measured in tt events. In single top quark events, the 
Wj combination that gives an invariant mass closest to the true top mass is chosen as the 
reconstructed top quark. In Zti events there are two W bosons and two 6-jets in the final 
state. Labeling the leptonically decaying W boson Wt and the hadronically decaying W- 
boson Wh, we loop over the combinations of the H^-bosons and 6-jets, i.e. (Wibi, Whb 2 ) and 
(W^&2, Wibx), and we calculate the invariant masses of the reconstructed top quarks. We 
then require all the masses of the reconstructed (Wb) systems to be within 20 GeV of the 



17 



top quark mass m t = 173.1 GeV [70|, i.e. 



m Wb — m t \ < 20 GeV. 



We calculate the deviations from the true top quark mass (m t ) for each combination, 



and select the combinations with the minimal deviations to be the reconstructed top quark 
pair. This simple algorithm works well. It picks the correct combination about 99% of the 
time. 

Our analysis is limited mainly by the neutrino reconstruction and experimental uncer- 
tainties. For example, due to neutrino reconstruction, the top quark reconstructed from 
the be + $t system exhibits a much broader mass spectrum compared to the anti-top quark 
reconstructed from three jet system of bjj, as seen in Fig. [31 

2. A(f)(l + £~) and Spin Correlations 

An observable is needed which changes shape in the presence of the anomalous couplings 
Tl and J-r. The opening angle between the two charged leptons from the Z boson decay Z — > 
£' + £'~ in the transverse plane, A<f)(£' + £'~), is such a candidate as is pointed out in Ref. jsj]. 
Since A0« pertains to the Z boson, it does not depend on the neutrino reconstruction. 
To illustrate this point, we plot the A0« distribution in the SM in Fig. H^a). Neutrino 
reconstruction reduces the number of observed events but does not change the shape of 
distribution. The kinematic cuts suppress the number of events but also do not change the 
shape. 

The sensitivity of A0« to the anomalous couplings is shown in Fig. IH(b) where we choose 
the somewhat generous values JF L = —0.1 and = 0.1 for illustration. Note that values 
of this magnitude are not inconsistent with the Fermilab collider data on single top quark 
production, mentioned at the beginning of this section, although for such large JF L and Tr 
there must be additional contributions, other than those from the operators considered in 
Section II, in the precision electroweak measurements so as to relax the bounds in Fig. [TJ 

The top quark spin asymmetry is another observable sensitive to the anomalous cou- 
plings. At the LHC Ztt production proceeds predominately through the gluon fusion pro- 
cess gg — > Ztt. The presence of left- and right-handed couplings of the Z to the top quark 
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FIG. 4: (a) The SM distribution of A(f>u, where the black curve denotes the distribution at the 
partonic level without any kinematic cuts, the red curve labels the distribution after neutrino 
reconstruction without kinematic cuts, the green curve represents the partonic distribution after 
kinematic cuts, and the blue curve shows the distribution after kinematic cuts and neutrino re- 
construction; (b) Ratio of the new physics cross section to the SM result in the A(f>g£ distribution, 
J~L = —0.1 and Tr = (black), and Tr = 0.1 and Tl = (red). The fluctuations are caused by 
statistics. 

in Zti production means that parity is slightly broken, resulting in a small top quark spin 
asymmetry. The anomalous couplings Tl and Tr might amplify or weaken the parity vio- 
lation effects, and a measurement of the asymmetry might provide a good probe for these 
anomalous couplings. Among the decay products of the top quark, the charged lepton is 
maximally correlated with the top quark spin 
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721 ] . We can thus obtain the most dis- 



tinctive distribution by plotting the angle between the spin axis and the charged lepton in 
the reconstructed top quark rest frame. Different choices of the reference frame to define 
the top quark polarization are found in the literature. In the helicity basis the top quark 
spin is measured along the top quark direction of motion in the center of mass (cm.) frame. 
However, one must bear in mind that the anomalous couplings affect the production of the 
Z boson and top quark pair but not the top quark decay. Therefore, these couplings can 
be probed better in the spin correlation between the top quark decay products and the Z 
boson decay products. We find that the choice of the negatively charged lepton from the Z 
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FIG. 5: The upper panels show the SM distribution of cosOhei (left) and cos9 op t (right) where the 
meaning of colored curves is the same as Fig. UJ The lower panels show the ratio of the new physics 
cross section to the SM value in the distributions of cos 9h e [ (left) and cos 9 op t (right) after kinematic 
cuts and neutrino reconstruction: black for Tl = —0.1 and J-r = 0, and red for Tr = 0.1 and 
Tl = 0. Here, Oheii^opt) is the angle between the positron and the top quark spin in the top quark 
rest frame in the helicity (optimal) basis, respectively. 

boson decay to measure the top quark spin direction amplifies the top quark spin correlation 
effects by almost a factor of 2. 

In Fig. [5] we show the cos 9 distribution for different spin bases, where cos 9 is defined as 

cos^ = A_^. (24) 
I stilus I 
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Here St is the three-momentum of the top quark spin in the reconstructed Zti cm. frame, 
and is the charged lepton three-momentum defined in the rest frame of the top quark. In 
the helicity basis s t is chosen to be the direction of the top quark in the cm. frame, while 
in the "optimal" basis s t is along the momentum of the negatively charged lepton from the 
Z boson decay in the top quark rest frame. 

The angular distributions in Fig. [5](a,b) show a clear slope before W reconstruction and 
kinematic cuts are imposed. However, the expected top quark spin correlations are diluted 
by the reconstruction of the neutrino momentum pz{v) and the kinematic cuts, as shown in 
Fig. [5j Affected significantly by kinematic cuts, the top quark spin asymmetry seems not 
the best choice of variable for measuring the anomalous couplings. It may be of relevance 
only once LHC experiments reach a very high level of accuracy. 



D. Projected Bounds 



1. LHC Reach 



We use the results of the event simulation outlined in Section IV B to derive projected 
bounds on deviations from the SM. The bounds are obtained here primarily from our fits 
to the distribution in A0«. Our simulation is done at the leading order level. Theoretical 
uncertainties, arising from the uncalculated higher order corrections and from the parton 
distribution functions, should be included in order to make a fully realistic prediction. Next- 
to-leading order (NLO) QCD corrections reduce the renormalization/factorization scale de- 
pendence to ~ 10% after the choice of an appropriate scale |60j, and they potentially affect 
final state differential distributions. While a detailed simulation at NLO is in order, it is 
beyond the scope of this work and we leave it to future work. Here, we conservatively con- 
sider an uncertainty of 30% on our Zti production estimates. The bounds will be improved 
when a more accurate NLO simulation is available. 

In Fig. [6] we display the deviation of the Zti cross section along one axis and the devi- 
ation of the single top quark production cross section along the other. Figure [6] (a) shows 
the projected 68% C.L. bounds on the Zti production cross section for integrated luminosi- 
ties of 300 fb -1 (solid) and 3000 fb -1 (dashed) at the LHC. The vertical bands denote the 
deviation of the single top quark production cross section: green for \8a t \ < 5% and blue 
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FIG. 6: (a): Projected 68.3% C.L. bounds on the deviation of the cross section for Zti production 
at the LHC with an integrated luminosity of 300 fb -1 (solid) and 3000 fb _1 (dashed) and from a 
LC (y/s = 500 GeV) with an integrated luminosity of 100fb _1 (magenta band). The expected 
accuracy of the single top quark cross section is represented by the vertical bands, green (blue) 
for 5<Jt < ±5(±10)%). (b): Various new physics model predictions, where the box (star, triangle) 
denotes the left-handed t' (right-handed t', sequential fourth generation) model, respectively. See 
text for details. 

for \5<j t \ < 10%. The straight lines in Fig. Mjo) demonstrate the strong correlation between 
Zti production and single top quark production induced by vanishing deviation of the Zbb 
coupling. The black solid line denotes Tr = 0, see Eq. (|2"T|) . Non-zero Tr will shift the 
curve up [Tr < 0) or down {Tr > 0), see blue (red) lines. 

The expectations of various new physics models are also plotted in Fig. 0(b). The top 
prime model 9] predictions are labeled by the (green) squares, where the mixing angle sl = 
0.3, 0.5, 0.6 from top to bottom corresponding to JF L = —0.045, —0.125, —0.18, respectively. 
We also consider a right-handed t' model [lOj] in which only the right-handed Zti coupling 
is modified. Its expectations are shown as the (purple) stars, where the mixing angle sr = 
0.8, 0.6, 0.4 from top to bottom corresponding to Tr = —0.32, —0.18, —0.08, respectively. 



Another model includes a sequential fourth generation [ll|| whose quarks mix substantially 
with the third family. Both the Zt^ii and the ZtRtR couplings are modified by mass mixing 
of the top quark and fourth generation up- type quark (w 4 ). One has to assume no mixing 
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of the bottom quark in order to protect the Zbb coupling. When u\ and u R are degenerate 
as it would be true in our effective operator framework, Tl = Fr, shown as the (black) 
triangles, sl = sr = 0.44, 0.63 [Tl = J~r = —0.1, —0.2) from top to bottom. 

Figure M is made under the assumption that the relation in Eq. ([TBI is valid. If instead 
one is interested checking Eq. (1131) as a prediction of SU(2)l x U(1)y symmetry, one has 
to measure Tl and Tr to a high precision from two independent measurements, in order 
to verify = 2^^^. Such measurements may have to await a LC where one can 

measure the top quark polarization by choosing the polarization of the incoming electron 
beam. Since the SM gauge symmetry is well-established so far, we simply invoke Eq. ( fl3l) 
to analyze data at the LHC One can still gain important information at the LHC: 

• Assuming 5V t b = 0, one can use the single top quark production measurement to deter- 
mine Tl- Then Tr can be determined from the Ztt total cross-section measurement 
uniquely. 

• If J-'l and Tr are measured from the differential distributions in Ztt production, one 
could then disentangle |Vth| from Tl in single top quark production. This result could 
allow us to determine whether 5V t b ^ 0. 

• The sign of the anomalous couplings as determined by the LHC data could carry 
important information in terms of distinguishing different classes of models. For ex- 
ample, if the anomalous couplings are induced by the mixing of SM particles with 
heavy exotic particles, then the sign is negative (relative to the SM coupling) due to 
the mixing matrix. Observation of a positive anomalous coupling would imply that 
either the new physics model is a strongly interacting theory or the third generation 



quarks are in a higher representation of the SM gauge group SU{2)l x U(1)y [25 ]. 



It is convenient to summarize the above results in the plane of anomalous Ztt couplings 
Tl and Tr as shown in Fig. The new physics models discussed above are distributed in 
different regions in the plot. 



2. Linear Collider Reach 

Here we comment briefly on the reach in a linear collider. The anomalous Ztt coupling 
could be measured in e + e~ — » 7/Z — > ti — ► b£ + i , bjj at a high energy electron-positron linear 
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collider. Our LHC results in Fig. [6](a) may be compared with expectations presented in the 



American Linear Collider working group report [73] for a 500 GeV machine and 100 fb _1 
luminosity. The electron beam is assumed to be 80% polarized. It is estimated that JF L 
and J~r can be measured to 3.7% and 3.2% accuracy in Ztt production, and that Tl can be 
measured to 2.5% in single top quark production. The magenta band in Fig. EJ^a) denotes 
the 1 standard deviation bound on the Ztt production cross section at a LC. Furthermore, 
a study of top quark anomalous coupling measurements via single top quark production 
at an electron-photon collider shows that Tl can be measured to 1% with 500 fb" 1 in a 



TeV machine 
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75] . The figures suggest that the LHC with C = 3000 fb 1 can reach 



a similar accuracy in Ztt production, together with precise measurement of the single top 
quark production cross section, as a LC (y/s = 500 GeV and C = 100 fb -1 ). 

As mentioned previously, such high precision at a linear collider could allow us to test the 
relation in Eq. (Tl3|) as a prediction of the electroweak symmetry, if one examines sufficiently 
many differential observables. In addition, the combination of data from the LHC and a LC 
may also allow for a precise determination of |V#,|, which would not be possible using single 
top quark production alone. 

Figure [7] shows that new physics cannot be tested or excluded beyond the 2a level without 
violating the bound from current low energy precision data discussed in Sec. Ill B[ But one 
should keep in mind that the low energy electroweak bounds could be diluted by additional 
tree-level induced operators. Our collider simulation is based on only two effective couplings 
and does not depend on four-fermion operators, making it less model dependent. If no other 
operators are present, a significant increase of linear collider energy and improvement of 
detector acceptance would be needed in order to test new physics via correlations between 
the Wtb and Ztt couplings. 



V. SUMMARY AND CONCLUSIONS 

In this paper we investigate correlations among the values of the Wtb, Zbb, and Ztt 
couplings. Two main contributions are made. First, we use a model-independent effective 
Lagrangian approach to parametrize the possible effects of new physics beyond the SM in 
terms of higher dimensional operators constructed from SM fields. We demonstrate that 
the SU(2)l x U(l)y symmetry of the SM yields correlations among the possible deviations 
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FIG. 7: (a): Projected 68.3% C.L. bounds on the anomalous Ztt coupling from the LHC with an 
integrated luminosity of 300 fb -1 (solid) and 3000 fb -1 (dashed) and from a LC (i/i = 500 GeV) with 
an integrated luminosity of 100fb _1 (magenta area). The expected accuracy of the single top quark 
production cross section is represented by vertical bands, green (blue) for Sat < ±5(±10)%). (b): 
Various NP model predictions, where box (star, triangle) denotes the left-handed t' (right-handed 
t', sequential fourth generation) model, respectively. See text for details. 

of the Zbifit, Zt L tL, and Wtj)) L couplings. Imposing the stringent experimental bound on 
Zbibi from data on Rb and A b FB , we show that a unique prediction follows on the Zt^L 
and WtibL couplings. The prediction is independent of the underlying new physics at the 
electroweak scale. We use existing experimental constraints to show that deviations of the 
Wtb and Ztt couplings from their SM values can depend on only two parameters, Tl and 
J-r, and we present the allowed ranges of these parameters for different new physics cutoff 
scales and Higgs boson masses. 

In the second contribution in this paper, we study the prospects for determining Tl 
and Tr with LHC data. We focus on single top quark and Ztt associated production. 
Deviations from the SM of the single top quark total and differential cross sections depend 
only on Tl, whereas the cross section for Ztt associated production is influenced by both 
Tl and Tr. We perform a detailed Monte Carlo simulation of the Ztt process, including 
the effects of experimental cuts. Among observables in Ztt that could be sensitive to the 
presence of Tl and Tr, we examine correlations between the top quark spin and the charged 
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lepton from t — > > £ + z/). We find that experimental cuts appear to leave a sample 

of effectively unpolarized top quarks, a disappointing conclusion that does not appear to 
have been established previously. Among the final state distributions in Zti associated 
production, we confirm that the opening angle between the two charged leptons from the 
Z boson decay, A(p(£' + £'~), seems to be useful for limiting the couplings Tl and Tr. The 
principal results of our analysis are presented in Figs. [6] and [7] as a set of two-dimensional 
plots of the deviation from the SM of the associated production cross section 8a zti versus the 
deviation of the single top quark cross section 8a t for integrated luminosities of 300 ftT 1 and 
3000 fb -1 at the LHC These expectations are contrasted with estimates for a high energy 
electron-positron linear collider (LC). A comparison is also shown with predictions based 
on a few recent models of new physics including a top-prime model [9|, a right-handed t' 
model [h]], and a model with sequential fourth generation quarks that mix with the third 
generation Our analysis suggests that about 3000 fb _1 at the LHC would be needed to 
improve model independent constraints on Tl and Tr to the same level achievable at a 500 
GeV LC with 100 ftT 1 of integrated luminosity. 

We also point out that the pattern of deviations predicted by the electroweak symmetry 
could allow us to separate the effect of the CKM element \V t i\ from that of the gauge coupling 
gwtbi which is also present in the Zti matrix element. Measurements of the Zti coupling 
could then be used as a constraint in analyses of data on single top quark production in 
order to determine 8V t b- A non-zero 8V t b would indicate non-unitarity of the 3x3 CKM 
matrix and be a clear signal for physics beyond the SM. 
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